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SIR: 

Now comes Jonathan Liou, who declares and states that: 

1. I am currently employed by MediaTek Inc. as a Patent Engineer in the LIP 
(Legal/Intellectual Property) group. I have been continuously employed by MediaTek Inc. since 
September 2006. Prior to that time, I was employed by the United States Patent and Trademark 
Office as a Patent Examiner in the area(s)/fteld(s) of multiplexing and data communications. 
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2. I received a Bachelor's degree in Electrical Engineering from the University of 
Pittsburgh in 2003, and a Master's degree in Electrical Engineering from the Ohio State 
University in 2004. 

3. I have read the above-identified application, I have also reviewed U-S. Patent No. 
6,055,330 to Eleftheriadis et al (hereinafter i< Eleftheriadis")> u - s - Pat - No - 5,606,371 to Klein 
Gunncwick (hereinafter "Klein Gunncwiek"), and U.S. Pat No. 5,223,926 to Stone (hereinafter 
"Stone"). 

4. I understand that the claims of the above-identified application have been rejected 
as being unpatentable over Eleftheriadis in view of Klein Gunncwiek* 

5. I understand that one independent claim of the above-identified application is 
directed to a method for allocating bits to encode a plurality of frames of an image sequence, 
each frame of said image sequence comprising a plurality of objects, said method comprising the 
steps of: 

(a) determining a target frame bit rate, TV***, for each of the frames in 
accordance with a quantizer scale for each object in the frame; 

(b) allocating said target frame bit rate among the plurality of objects in 
accordance with the formula: 

where V» is a target object bit rate for each object, and Ki is proportional to an average 
pixel value for the object; 

(c) generating the quantizer scale for each of said plurality of objects in 
accordance with said target object bit rate, wherein said quantizer scale provides coarser 
and/or fewer allowed quantization values for a high frequency subband of said image 
sequence than for a low frequency subband of said image sequence; and 

(d) recursively adjusting the target frame bit rate for subsequent frames in the 
sequence* 

Page 2 of 11 
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6. I understand that another independent claim of the above-identified application is 
directed to an apparatus for encoding frames of an image sequence, each frame comprising a 
plurality of objects, said apparatus comprising: 

a motion compensator for generating a predicted image of a current frame; 

a transform module for applying a transformation to a difference signal between 
the current frame and said predicted image, where said transformation produces a plurality of 
coefficients; 

a quantizer for quantizing said plurality of coefficients with at least one quantizer 
scale for each object in the frame; and 

a controller for generating the at least one quantizer scale for each of said plurality 
of objects in accordance with a target object bit rate, wherein said quantizer scale(s) provide 
coarser and/or fewer allowed quantization values for a high frequency subband of said image 
sequence than for a low frequency subband of said image sequence, selectively adjusting said at 
least one quantizer scale for the current frame in response to said target object bit rate for each of 
the plurality of objects, and determining said target object bit rate from a target frame bit rate, 
Tfr«n», in accordance with the formula: 

Vi»K|XTW 

where Vj is a target object bit rate for each object, and Kj is proportional to an average pixel 
value for the object 

7. I understand that a third independent claim of the above-identified application is 
directed to a computer-readable medium having stored thereon a plurality of instructions which, 
when executed by a processor, perform steps comprising: 

(a) determining a target frame bit rate, Tiram* for a frame in an image 
sequence in accordance with a quantizer scale for each object in the frame, wherein said 
each frame includes a plurality of objects; and 
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(b) allocating said target frame bit rate among the plurality of objects in 
accordance with the formula: 

where V; is a target object bit rate for each object, and K, is proportional to an average 
pixel value for the object; 

(c) generating the quantizer scale for each of said plurality of objects in 
accordance with said target object bit rate, wherein said quantizer scale provides coarser 
and/or fewer allowed quantization values for a high frequency ^subband of said image 
sequence than for a low frequency subband of said image sequence; and 

(d) recursively adjusting the target frame bit rate for each frame in the 
sequence* 

8* Eleftheriadis discloses a method and apparatus for performing digital image and 
video segmentation and compression using 3-D depth information (Title), In contrast to 
paragraphs 5-7 above, which recite allocating the target object bit rate(s) m accordance with the 
target frame rate and an average pixel value for the object, Eleftheriadis appears to determine a 
target object bit rate based on a quantizer (the value of which appears to be related to the distance 
of the object from the camera; see, e.g»> col. 1 1, 1L 1-15 and 41-44) and the proportion of pixels 
in the object (see, e«g», col. 11,1. 65-coL 12, 1. 10; emphasis added), 

9. Thus, Eleftheriadis is deficient with respect to the method, apparatus and 
computer-readable medium of paragraphs 5*7 above. 

1 0. Eleftheriadis does not disclose allocating a target frame bit rate among the objects 
in a frame according to an average pixel value for the object, nor does Eleftheriadis et al. appear 
to disclose the present step of generating a quantizer scale for each object that provides coarser 
and/or fewer allowed quantization values for a high frequency subband than for a low frequency 
subband of the image sequence (see paragraphs 5-7 above). While Eleftheriadis mentions the 
quantization of pixels and frames that comprise a plurality of objects, one benefit of the present 
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claimed method is that the target bit rates can be prioritized based on the relative need of the 
objects for the available bits (as reflected by the average pixel value) in a given application, 
rather than on the size of the objects. 

11. Furthermore, Eleftheriadis does not disclose a controller for generating such a 
quantizer scale and determining the target object bit rates from a target frame bit rate in 
accordance with an average pixel value for the objects (see paragraph 6 above). 

12. Allocating a target frame bit rate among the objects in a frame according to an 
average pixel value for the object is advantageous because it provides more efficient use of 
encoder bandwidth relative to determining an object bit rate according to the frame bit rate and 
the number of pixels in the object, as taught by Eleftheriadis. 

13. In combination with generating the quantizer scale for each object in accordance 
with the target object bit rate, allocating a target frame bit rate among the objects in a frame 
according to an average pixel value for the object enables objects of interest to receive a higher 
bit rate than objects of lesser interest, regardless of the number of pixels in (or size of) the object. 

14. One advantage of the method is that an object having a smaller number of pixels, 
but needing more bits (e,&> in terms of encoding syntax information, motion information and/or 
shape information; see, e.g., page 14, lines 2-7 of the present specification) can have a greater 
proportion of the available bandwidth, or target frame bit rate, than a larger object that does not 
need as many bits. In contrast* the approach of Eleftheriadis appears to assign a certain 
proportion of the available frame bandwidth based solely on the size of the object, without 
reference to the relative need for encoding bits by the various objects in the frame. 

15. Klein Gunnewiek is silent with regard to allocating or determining an object bit 
rate according to an average pixel value of the object Thus, Klein Gunnewiek fails to cure this 
deficiency of Eleftheriadis regarding the method, apparatus and computer-readable medium of 
paragraphs 6-8 above. 
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16. Neither Eleftheriadis nor Klein Gunnewiek disclose quantizing high frequencies 
more coarsely and/or with fewer allowed values than low frequencies or the possible benefits 
thereof 

17. Eleftheriadis also does not allocate a frame bit rate among objects in the frame, as 
recited in paragraphs 5-7 above. Instead, Eleftheriadis determines object bit rates, adds them up, 
then adjusts the total frame bit rate based on buffer fullness (see the discussion below), Thus, 
Eleftheriadis takes a "bottoms up^-type approach to determining object and frame rates* whereas 
the method, apparatus and medium of paragraphs 5-7 above use a "tops down n -type of approach, 

18. Eleftheriadis discusses two coding techniques for controlling bit rates, variable bit 
rate (VBR) coding and constant bit rate (CBR) coding (sec col- 8, 11. 8-19 and col. 11, 1L 39-64). 
Eleftheriadis discloses a constant bit rate encoder (FIG. 10), in which an object map generated by 
object segmentation circuit 500 is received by a macroblock labeling circuit 1 100 (sec col. 1 0, 1. 
65-col. 11,1. 1). Since the encoder splits each frame of video information received from the 
camera into macroblocks and quantizes DCT coefficients on a macroblock basis, Eleftheriadis 
teaches that it is desirable to assign each macroblock of video data to a specific object, or in the 
case of a simple segmentation technique described therein, to a region which contains one or 
more objects at the same depth from the camera (col. 11, II. 1-8), Once the macroblock 
including pixels from an object or region has been assigned, it will be assigned to one object or 
region by macroblock labeling circuit 1100, a rate controller 1040 can select an appropriate 
quantizer step size for the entire current macroblock (col. 1 1, U. 12-15). 

19. For CBR coding, the rate controller 1040 must additionally regulate quantizer 
selection so that the output buffer 1020 neither overflows nor underflows. Since the total 
number of bits per second which may be output is nowfixed* object sizes become important 
(col. 1 1, 11. 53-57; emphasis added). In accordance with a known technique for performing area- 
selective rate control when the object locations are known, each object is associated with a 
particular target average bit rate Rj» except for the background (object n). Thus* in CBR coding, 
Eleftheriadis appears to determine a target object bit rate based on a quantizer value, rather than 
an allocation in accordance with the target frame rate, as recited in paragraphs 5-7 above. 

Page 6* of 11 
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20, In order to maintain the given total average rate R necessary to prevent buffer 
overflow, the background rate is determined according to EQ. (4): 



where cti is the proportion (from 0.0 to 1.0) of the pixels in the frame that belong to object i (col. 

11, L 67-coL 12, L 10; emphasis added). 

21. Thus, Eleftheriadis clearly determines a target object bit rate based on the 
proportion of pixels in the object, rather than on average pixel value for the object, as explained 
above. 

22. Elefthcriadis explicitly teaches that it is possible that R„ (the background bit rate) 
is negative (coL 12, I. 16). To one of ordinary skill in the art this possibility of a negative 
background bit rate demonstrates that target object bit rates are not allocated in accordance with 
a target frame rate. Rather, they must be determined by some other technique (such as in 
accordance with a quantizer value, as explained above, which in turn appears to be based on the 
occupancy of a buffer [col. 13, 11. 22-26], the output rate of which is constant and dependant 
on the bandwidth of the channel which is accepting data from the buffer; see col. 1 3, 11. 1 7-22). 

23. Eleftheriadis further teaches that the possibility of a negative background bit rate 
may simply have the effect of assigning as coarse quantization as possible to the background, 
and may result in less average bits per second per object than the target bit rates Ri indicate (coL 

12, 1. 16-19), further confirming that target object bit rates therein are not allocated in 
accordance with a target frame rate, 

24- Thus, there appears to be no reasonable basis in Eleftheriadis for an assertion that 
EQ- (4) of Eleftheriadis is used to determine the target frame rate; to the extent R may be related 
to a target frame rate, Eleftheriadis teaches that it is given (e.g., determined a priori, and 
dependent on the bandwidth of the channel which is accepting data from the buffer). 
Furthermore, the rate control performed by placing a buffer 320 at the output 310 of the variable 
bit rate (VBR) encoder 200 and having a rate controller 340 takes into account the occupancy of 
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the buffer and other parameters that are possibly signal dependent in order to decide the 
quantizer step size in quantizer 251 , so that the buffer does not overflow or underflow (coL 8. H. 
17-27, and Fig. 2-3). Consequently, in the relevant discussion of CBR coding, Eleftheriadis feils 
to disclose step of allocating the target frame bit rate in accordance with a target object bit rate as 
recited in the method, apparatus and computer-readable medium of paragraphs 5-7 above. 

25. Also, as discussed above, since Eleftheriadis teaches that the frame bit rate is 
related to the occupancy of a buffer having a constant output rate that is* in turn, dependent on 
the bandwidth of the channel which is accepting data from the buffer, Eleftheriadis cannot 
disclose or suggest to one of ordinary skill in the art that the frame bit rate can be (much less 
should be) adjusted. Thus, Eleftheriadis appears to be silent with regard to recursively adjusting 
a targe* frame rate, as recited in paragraphs 5 and 7 above. 

26. For VBR coding* Eleftheriadis teaches that macroblock labels can be directly used 
for rate control by associating particular quantizer step sizes with each object (col. 11, 11. 39-41). 
The encoder can also employ techniques to "smooth out" quantizer differences at object 
boundaries by gradually changing the quantization step while entering or exiting an object (col. 
11, 11. 47-50). A macroblock labeling circuit 1100 (see FIG* 11) contains object identifications 
for each pixel in the macroblock (col 11, 11. 16-19), and quantizer selection is simply a lookup 
operation into a table which indexes the possible object identifications generated by macroblock 
labeling circuit 1100 (col, 11, II. 44-47). Also, Eleftheriadis teaches that rate control is also 
usable in a purely VBR encoder to provide higher quality for some image areas, and less for 
areas that have smaller significance (e.g., background areas). As a result, the term rate control is 
used by Eleftheriadis generally without discriminating whether or not a CBR or VBR encoder is 
used (coL 8, 11. 40-47). Thus, the discussion of VBR coding by Eleftheriadis fails to cure any 
deficiency of Eleftheriadis with regard to the allocating step in paragraphs 5 and 7 above* 

27. Consequently, Eleftheriadis fails to disclose allocating the target frame bit rate 
among a plurality of objects in the frame in accordance with an average pixel value for the object, 
as recited in paragraphs 5-7 above. 
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28. Allocating the target frame bit rate among a plurality of objects in the &ame 
provides for more efficient use of encoder bandwidth and reduces the risk of buffer overruns, 
relative to the approach of Eleftheriadis (which can assign or allow a total of the object bit rates 
that i$ greater than the total bit rate for the frame). Reducing the risk of buffer overruns also 
reduces the probability of skipping frames, which can affect the quality of a digital video picture 
stream. 

29. Furthermore* Eleftheriadis fails to disclose the controller of paragraph 6 above, 
which determines the target object bit rote from a target frame bit rate in accordance with an 
average pixel value for each of the objects. As discussed above, Eleftheriadis determines a target 
object bit rate based on the proportion of pixels in the object, rather than an average pixel value 
for the object In addition, Eleftheriadis discloses a rate controller 1040 that regulates quantizer 
selection so that the output buffer 1020 neither overflows nor underflows (col. 1 1, 1L 53-57). 
However, the output rate of the buffer is constant and dependant on the bandwidth of the channel 
which is accepting data from the buffer (see col, 3, IK 17-26). Eleftheriadis does not appear to 
disclose any connection between the bandwidth of the channel which is accepting data from the 
buffer and a target frame bit rate. As a result, Eleftheriadis is deficient with regard to the method 
described in paragraph 5 above* 

30. Klein Gunnewiek discloses a device for encoding a video signal comprising 
means for dividing each picture into a plurality of sub-pictures, an encoder comprising a picture 
transformer for transforming each sub~picture into coefficients, and a quantizer for quantizing 
the coefficients with an applied step size (col. 1, II. 5-10). Klein Gunnewiek neither discloses 
nor suggests allocating a target frame bit rate among the objects) in a frame according to an 
average pixel value for the object, nor does Klein Gunnewiek disclose or suggest generating a 
quantizer scale for each of the objects in accordance with the target object bit rate, the quantizer 
scale providing coarser and/or fewer allowed quantization values for a high frequency subband 
than for a low frequency subband of the image sequence. 

31. There is no reasonable basis for interpreting cither cited reference as disclosing a 
target object bit rate determined and/or based on an average pixel value. The constant values Kf> 
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and K B of Klein Gunnewiek arc not average pixel valves* regardless of how they may be applied 
in an equation. Instead, they are constants that are related only to the gain of the encoder and the 
quantizer step size (col 3, H. 1 1-15, and col 4, h 44<ol. 5, 1. 10 of Klein Gunnewiek). Thus, the 
constant values Kp and K B of Klein Gunnewiek are constants of the system, and are arguably not 
related to pixel values at alt 

32. As a result, the average pixel value K t as recited in paragraphs 5-7 above has a 
different nature (e.g^ it is a variable) than Kp and Kb of Klein Gunnewiek, and it is not an 
obvious design choice or variation to one of ordinary skill in the art to substitute an object-based 
variable for a constant of the system. 

33. Thus* Klein Gunnewiek foils to cure the deficiencies of Eleftheriadis with respect 
to allocating the target frame bit rate among the objects in each frame in accordance with an 
average pixel value for the object, as recited in paragraphs 5 and 7 above. Similarly, Klein 
Gunnewiek fails to cure the deficiencies of Eleftheriadis with respect to a controller for doing the 
same, as recited in paragraph 6 above. Consequently, no possible combination of Eleftheriadis 
and Klein Gunnewiek can suggest the method, apparatus and computer-readable medium of 
paragraphs 5-7 above. 

34. Stone does not cure the deficiencies of Eleftheriadis and Klein Gunnewiek with 
regard to the method, apparatus and computer-readable medium of paragraphs 5-7 above. 

35. Stone discloses compressing a digital video signal by spatial sub-band filtering to 
form data sets constituting respective sub-bands of the two-dimensional spatial frequency 
domain (Abstract, 11. 1-3). The data sets for a field or frame are stored, and a first sequencer 
controls writing the stored data to a quantizer in which they are quantized in accordance with 
respective values, those values being such that the amount of quantization of at least a data set 
constituting a sub-band to which dc luminance information of the signal is at least predominantly 
confined is less than the average of the amounts of quantization of the remaining data sets 
(Abstract, 1L4-12). 
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36. Stone further discloses intra-image compression in the time domain by the use of 
differential pulse code modulation, in which a predictor is used to predict the values of samples 
representing pixels based on previous pixel values because image pixels are highly correlated, 
and the small and uncorrected error can be encoded using fewer bits than the samples 
representing the original pixels (col. 1, 11. 27-37). Stone suggests that a selective quantization 
operation and/or entropy encoding in accordance with the description therein can lead to bit rate 
reduction (col. 15, 1L 45-52; col. 18, 1L 39-46; and coL 20, 11. 24-46). 

37. Stone is silent with regard to (a) allocating the target frame bit rate in accordance 
with a target object bit rate and/or (b) allocating a target frame bit rate among the objects in each 
frame in accordance with an average pixel value for the object, as recited in paragraphs 5-7 
above. 

38. Thus, Stone fails to cure the deficiencies of Elefiheriadis and Klein Gunnewiek 
with respect to the method, controller and computer-readable medium of paragraphs 5-7 above. 
Consequently, no possible combination of Eleftheriadis, Klein Gunnewiek and Stone can suggest 
the method, apparatus and computer-readable medium of paragraphs 5-7 above. 

Further, Declarant sayeth not 

I hereby declare that all statements made herein of my own knowledge are true and that 
all statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made arc 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 
Code and that such willful false statements may jeopardize the validity of the above-identified 
application or any patent issued thereon or therefrom. 
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